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APPARATUS FOR GENERATING CONTENT CODES 
FOR AUDIOVISUAL PROGRAMS 



BACKGROUND OF THE INVENTION 

A. Field of Invention 

This invention relates to the tagging of audiovisual content of a 
program, such as analog or digital motion pictures that are transmitted over 
cable, or video transmitted over the Internet More particularly, it pertains to a 
viewer-operated apparatus for tagging individual scenes of the audiovisual 
program so that it can later be replayed with selected scenes being omitted 
or replaced by others. 

B. Description of the Prior Art ! 

Presently, many video programs have ratings that can be used by 
viewers to determine whether they would like to play the programs, or whether 
they would allow others (such as children or adolescents) to play them. 
These types of ratings are also becoming available for TV shows, games, and 
other audiovisual programs. Programs broadcast over the Internet may be 
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filtered or otherwise restricted using on-line filtering services, and set-top 
boxes and TVs with V-CHIP content filtering technology also exist for this 
purpose. 

As an improvement to current technology, It has been suggested that 
rating codes be provided for individual segments, or scenes, of programs. 
Automated playing devices would then be able to seamlessly eliminate 
segments with selected content, and/or substitute other segments with 
acceptable content if such segments are available. Commonly assigned co- 
pending application S.N. 09/479,819 filed January 7, 2000, entitled M Content 
Control of Broadcast Programs" and incorporated herein by reference, 

discloses a personal video recorder that receives and stores a broadcast 

I 

program with each scene being tagged with a content rating. Parameters for 
acceptability of content are designated by viewers, and the appropriate 
scenes are played as a seamless continuous program. U.S. Patent Nos. 
5,987,21 land 6,208,805 to Abecassis discloses an apparatus which records a 

program on a disk and filters content according to viewer preferences. A 

I 

viewer is shown only the segments having acceptable content. A parent has 
the ability to review a program and tag unacceptable scenes. The apparatus 
then omits the unacceptable scenes when the program is played, 

A major-disadvantage of all of these systems is that, whether a 
universal rating is assigned to a whole program, or ratings are assigned to 
each of a program's individual scenes, the ratings still reflect a limited number 
of categories accepted by the public at large, industry-wide guidelines or the 
opinions of an editor. Thus, they fail to reflect inherent differences in the 
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filtering categories of individual viewers due to many factors such as 
geographic location, level of education, income, religious and moral beliefs, 
etc.,- factors that meaningfully affect a person's views and preferences for 
what kind of programs and content they, or their children, should view. 

OBJECTIVES AND SUMMARY OF THE INVENTION 

It is believed to be beneficial to allow viewers to tag program scenes in 
keeping with their own preferences. Many programs, e.g. many TV shows, 
and unrated movies, lack any form of scene tagging, and in other instances 
adults may desire to personally oversee audiovisual programs viewed by their 
children. 

Accordingly, it is an objective of the present invention to provide an 
apparatus that permits viewing of scenes of a program, and the tagging of 
each scene with a content code that can be used later for filtering purposes. 

A further objective is to provide an apparatus that is viewer-friendly and 
easy to operate, an apparatus capable of being operated at home by an 
ordinary consumer. 

Yet another objective is to provide an apparatus that can provide types 
of content tags other than conventional ratings. 

Yet another objective is to provide an apparatus that can be easily 
adapted to accept content tags for each scene. 

A further objective is to' provide an apparatus in which a program's 
scenes can be designated by the program provider, automatically detected, or 
designated by a viewer. 
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Briefly, a viewer displays a program on an audiovisual monitoring 
system. The apparatus for assigning content codes to the program's scenes 
constructed in accordance with this invention includes a scene detector 
identifying the individual scenes of a program. The scene detector, in one 
5 embodiment, detects scene changes by analyzing the characteristics of 
sequential frames using known techniques. In another embodiment, the 
program itself includes codes that identify Its scenes. In a third embodiment, 
the scene detector responds to a scene selector that is viewer activated. 

Also included in this invention is a scene indicator, which may be in the 
10 form of a visual or audible signal, and which indicates the beginning of a new 
scene. The viewer is prompted by the scene indicator to enter a content code 
for the new scene, this content code being related to the audio and visual 
content of the scene. In one embodiment, once the content code is entered, 
the scene indicator is deactivated until the detection of the next scene. If the 
15 duration of the scene is known, a timer may be provided to indicate to the 
viewer the time left to the end of the scene. If no content code is entered by 
the viewer by the end of the scene, a default content code is automatically 
assigned for that scene. The default may be the code for the immediately 
preceding scene. In another embodiment, the scene indicator is activated 
20 only for a brief duration (e.g., 5-10 seconds) to indicate the beginning of each 
scene (and, implicitly, the end of the previous scene). 

A code selector is also included in the apparatus. This code selector is 
adapted to receive the content code for a scene and to enter into a file a two- 
component code (two record fields) for the scene: an identification of the 
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scene and the content code for the scene, the identification may be 
generated by a timer that times the duration of each scene as well as previous 
scenes and then generates a starting and ending time for the. scene. The 
apparatus also includes a memory for storing the codes for all the scenes 
5 including the assigned content codes and scene identifications. 

The apparatus further includes a controller adapted to selectively 
activate the scene indicator and generally manage the generation and storage 
of the scene codes. 

In this manner, the apparatus described above provides a user with 
10 the ability to designate and assign content codes for the scenes of a program 
in accordance with his individual tastes or preferences, 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 shows a block diagram of an apparatus for assigning content 
15 codes to scenes in accordance with this invention; 

Fig. 2 shows a flow chart for the apparatus of Fig. 1 ; and 
Fig. 3 shows a somewhat diagrammatic view of a selector panel on 
which a viewer can enter his preferences. 

20 DETAILED DESCRIPTION OF THE INVENTION 

Referring to Fig. 1, an apparatus 1 0 constructed in accordance with the 
invention includes a device such as a program player 12 that is adapted to 
play an audiovisual program on an audiovisual monitoring system 14. The 
program could be recorded on a DVD disk, a VCR tape or other similar media, 
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or in a personal video recorder receiving the program from a program source 
16, e.g. through a cable, XDSL, or satellite connection. 

Associated with the player 12 are a scene detector 18 and a program 
timer 20. The scene detector 18 is used to detect seeftdTln the program 
» signal generated by player 12. The program timer 20 generates a count that 
can be used to provide unique identifications for the scenes of the program. 
Instead of the timer, other scene identification means may be used, such as a 
frame counter. 

A controller 22 directs the operation of the apparatus 10. The controller 
22 is coupled to the scene detector 18 and timer 20. and it is also associated 
with a code selector 24, a scene indicator 26, a scene timer 27, and a 
memory 28. It should be understood that all the elements in Fig. 1 can be 
implemented as a microprocessor-based device such as a PC, or can be 
incorporated into the program player 12 (especially if the player is a personal 
video recorder). However the apparatus 10 is shown in Fig. 1 as having 
discrete components for the sake of clarity. 

The apparatus of Fig. 1 operates as indicated by the flow chart of Fig. 
2. In step 100 the program player is activated to start playing a program on 
audiovisual monitoring system 14. The program timer 20 is activated as well. 

As the program is shown, the frames forming (the program are 
monitored by scene detector 1 8 to detect the start of scenes that comprise 
the program. Scene detectors of this kind are well known in the art. When 
the first scene is detected (step 1 02), the controller 22 logs its starting time as 
indicated by program timer 20 (step 104). 



5 



W ° 03/012583 PCT/US02/23714 

Next, the scene indicator 26 is activated (step 1 06). The scene 
indicator 26 may be a simple light that is' turned on or a beeper that is 
activated. The scene indicator 26 is activated to prompt the viewer for a 
content code, using the code selector 24. The code selector 24, shown in 
more detail in Fig. 3, consists of a housing 30 which supports a plurality of 
switches 32, 34, 36 and 38. Each of the switches may be labeled with a 
content code, for example G, PG, R or X (More precise cbntent codes may 
be used, giving rise to more than just four switches). For example, the 
content codes may indicate that a scene includes profanity, violence, sex, 
io nudity, etc. Alternatively, the content codes may be selected using various 
other criteria. For example, if the program is a documentary about nature, the 
content code may be used to designate all scenes showing animals. Each of 
the switches 32-38 may be a momentary pushbutton switch. The scene 
indicator 26 may be positioned on the housing 30 as well. 
15 ln some instances, the scene detector 18 may be too sensitive for the 

purposes of the present invention. More particularly, the scene detector may 
designate every camera angle change as a new scene. Therefore it may be 
advisable to have a viewer determine the start and end time of each scene. 
For this purpose, the code selector 24 may include a manual scene selector 
20 40. For example, the manual scene selector 40 may be a momentary 
pushbutton like switches 32-38. When this manual scene selector 40 Is 
provided, the scene detector 18 can be turned off or omitted. 

In such an embodiment, after the program starts a viewer selects the 
start time of a scene by activating the manual scene selector 40. When the 
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manual scene selector 40 is activated, the scene indicator 26 is also 
activated, thereby indicating that the current scene has not yet been assigned 
a content code. The start time is also logged, as in the embodiment of Fig. 2. 
When the viewer selects a content code, the indicator 26 goes off and stays 

5 off until the next scene is identified by the viewer operating button 40. In an 
alternate embodiment of the invention, the scene indicator 26 is activated only 
for a brief time, (e.g., three to ten seconds) duration to alert the viewer of the 
beginning of a scene (and the end of the previous scene). In this 
embodiment, the scene indicator 26 is deactivated independently of the 

io selection of a content code by the viewer. 

As discussed above, as each scene is displayed on audiovisual 
monitoring system 14 with audio heard through associated speakers (not 
shown), the scene indicator 26 is activated, As the viewer views and listens 
to the current scene he makes a determination of what its content code 

is should be. He then designates the content code for the scene by activating 
the appropriate switch 32-38. (It is possible to activate multiple switches If the 
system accommodates multiple content codes for a scene.) 

Returning to Fig. 2, in step 108,the controller 22 checks if the viewer 
has yet selected a content code. If a content code has been selected, then in 

20 step 1 10 the content code is logged together with a corresponding scene 
identification, e.g., its starting time code. In step 1 12 the controller 22 
deactivates the scene indicator 26. (If the scene indicator 26 is a beeper, it 
may emit a different sound than when It was activated in step 106). 

In step 114 the controller 22 checks the scene detector 18 for a new 
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scene. When a new scene is detected, the controller 22 logs the end (e.g., 
another time code) of the last scene (step 116). 

Next, in step 118 scene indicia, including the appropriate content 
code(s) and the corresponding scene identification data, are stored in scene 
content code memory 28. 

In step 120 a test is performed to determine 'if the end of the program 
has been reached. If the program end has not been reached, then the start 
time of the new scene is logged in step 104 and the new scene is processed 
as discussed above. 

In step 120 if the program end is detected, then In step 122 all the 
scene indicia are recorded in a specific file that may be left in memory 28, 
stored on the medium which itself contains the program, or somewhere else. 

Back at step 108, if a content code has not been selected yet for the 
current scene, then in step 1 24 the controller 22 checks the scene detector 1 8 
for a new scene. If a new scene is not detected, then the system 22 goes 
back to step 108. If a new scene is detected in step 124, then in step 126 a 
default content code is assigned to the scene. For example, the default code 
may be the code assigned to the previous scene. 

In step 128 the end of the scene being processed is logged and the 
process continues with step 1 1 8. 

The apparatus 10*'can be modified for other types of operation as well. 
In one embodiment the scenes may be already defined by a specific code 
embedded within the program itself. In this case, instead of a scene detector, 
a decoder detects the beginning and end of each scene. This alternative is 
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indicated in Fig. 1 for element 18. ' 

If the beginning and end points of each scene (and therefore Its 
duration) are known, then, at the beginning of each scene, a scene timer 27 
may also be activated. This scene timer 27 can be configured to count down 
5 and indicate to a viewer "how much time is left to the end of the scene (i.e., the 
time left before the viewer has to select a code). 

Instead of using an automatic or even manually operated scene 
detector, it is possible simply to allow the viewer to enter content codes at any 
time. Such changes in content codes would In effect signal a new scene. 

10 The advantage of identifying the start of a scene, however, is that the entered 
content codes are retroactive to the start of the scene. On replay, an 
appropriate player could curtail viewing of an objectionable scene from its 
start, not just from when the content codes were entered. 

By generating a content code for each scene of a program, a table is 

15 created which associates each scene with its content code and the address of 
the next scene. This table may be stored separately or may be attached to 
the program itself. The table can be used to control the manner in which the 
program is played. For example, co-pending commonly assigned U.S. 
application S.N. 09/479,819 filed January 7, 2000, entitled CONTENT 

20 CONTROL OF BROADCAST PROGRAMS and incorporated herein by 

reference discloses a method and apparatus wherein a program is broadcast 
together with its table to a customer with an appropriately configured personal 
video recorder. The customer selects an acceptable content code. The 
personal video recorder then can record and play only the scenes that have a 
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predetermined content code, while skipping the scenes with other codes. 
Since the table with the content codes is received in advance, the personal 
recorder can determine the scenes that are to be skipped, and the duration of 
these skipped scenes. The recorder therefore can delay playing the program 
by the duration of the skipped scenes so that the program can be played 
without interruption. 

Another advantage of providing a program with a table of scene- 
associated content codes is that it allows a content provider to create a 
program with several interchangeable scenes, each scene having a unique 
content code. A customer may then elect to get a program with all the 
scenes. Alternatively, a customer may elect to a version of the program with 
all the scenes having a predetermined content code or codes. 

Obviously, numerous other modifications can be made to this invention 
without departing from its scope, as defined in the appended claims. 
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I claim: 

1 . An apparatus for allowing a viewer to assign content codes to scenes in an 
audiovisual program comprising: 

a device for playing the program and provided with a scene indicator to 
identify a new scene; 

a content selector having a viewer input for generating a content code 
for a scene; and 

a memory for storing content codes associated with respective scenes. 

2. The apparatus of claim 1 wherein said memory stores data relating to the 
beginnings and endings of scenes together with respective content codes. 

3. The apparatus of claim 2 further comprising a scene indicator that is 
activated when a new scene is played. 

4. The apparatus of claim 3 wherein said scene indicator is deactivated after 
a content code for the scene is generated. 

5. The apparatus of claim 3 further including a scene detector for 
automatically activating said scene indicator. 

! 

6. The apparatus of claim 3 further including a manually operated switch for 
activating said scene indicator. 

i 
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7. The apparatus of claim 1 further comprising a scene indictor that is 
activated when a new scene is played, 

8. The apparatus of claim 7 wherein said scene indicator is deactivated after 
a content code for the scene is generated. 

9. The apparatus of claim 7 further including a scene detector for 
automatically activating said scene indicator. 

10. The apparatus of claim 7 further including a manually operated switch for 
activating said scene indicator. 

11. The apparatus of claim 1 further including a timer to indicate the time left 
before the end of the scene. 

12. An apparatus for generating content codes for a program formed of a 
sequence of scenes comprising: j 

a device adapted to play said program in sequence for on an 
audiovisual monitoring system; 

a scene detector coupled to said player and adapted to detect a 
starting point for each of said scenes being played by said player; 

a scene indicator that is activated at the beginning of a scene requiring 
a content code; and 

a selector having an input to receive from a viewer a content code for 
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13. The apparatus of claim 12 wherein said player is adapted to play one of a 
video tape, a DVD disk and a broadcast program. 

i 

14. The apparatus of claim 12 wherein said scene indictor is deactivated 
when a content code is employed to the corresponding scene. ■ 

15. The apparatus of claim 12 wherein said program includes scene 
identification codes identifying the starting points of the scenes, wherein said 
scene detector is adapted to detect said scene identification codes. 

16. The apparatus of claim 12 wherein said scene detector includes a scene 
selector adapted to be activated by the viewer to indicate the starting point of 
said scene. 

17. The apparatus of claim 12 further comprising a controller adapted to 
determine if the viewer fails to provide said content code, said controller being 
adapted to generate in this case a default content code for said scene. 

18. The apparatus of claim 12 further comprising a timer indicating the time 
left before the end of the scene. 

19. The apparatus of claim 12 wherein said scene indicator is activated for a 
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predetermined time. 

20. The apparatus of claim 12 wherein said scene detector is deactivated 
when a content code is assigned to the respective scene. 

21. A method to be practiced by a viewer for generating content codes for the 
sequential scenes of a program comprising: | 

displaying said program to a viewer; 
identifying a scene of saicj program; 

obtaining a content code from said viewer regarding said scene; and 
storing said content code in a memory. 

22. The method of claim 21 further comprising storing in said memory data 
relating to the beginnings and endings of scenes together with said content 
codes. 

23. The method of claim 21 further comprising detecting a new scene and 
providing an indication thereof. 

24. The method of claim 23 further comprising deactivating the indication of a 
new scene after a content code is obtained thereof. 

25. The method of claim 23 further comprising said detecting each new 
scene and automatically generating said scene indication after the new scene 
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26. The method of claim 25 wherein new scenes are detected automatically 
by a scene detector. 

I 

27. The method of claim 25 wherein new scenes are determined by a viewer 
and new scen'e indicators are controlled by a manual switch operated by the 
viewer. 

28. The method of claim 21 further comprising generating a scene indication 
indicative of the beginning of each scene. 

29. The method of claim 28 further comprising deactivating said scene 
indication when a content code is assigned to the respective scene. 

30. The method of claim 28 further comprising deactivating said scene 
indication after a predetermined time. 

31 . The method of claim 28 further comprising providing a timer indicating the 
time left to the end of the scene. 

32. The method of claim 21 further comprising storing data related to the 
beginnings and endings of said scenes. 



WO 03/012583 PCT/US02/23714 



10 



INTERNET 
CONNECTION 
16 




PROGRAM 
PLAYER 
' 12 






o- 



SCENE 
DETECTOR 
(DECODER) 
18 




CONTROLLER 
22 



MONITOR 
14 




PROGRAM 
TIMER 
20 



SCENE 
INDICATOR 
26 




CODE 
SELECTOR 
24 



SCENE 
CONTENT 

CODE 
MEMORY 
28 



FIG, 1 




SCENE 
INDICATOR 28 



SCENE 
TIMER 27 



30 







MANUAL • 
SCENE SELECTOR 
• 40 



FIG. 3 



1/2 



WO 03/012583 



PCT/US02/23714 



ACTIVATE PLAYER 
(START TIMER) 



100 



DETECT FIRST 
SCENE 



102 



LOG STARTING 
TIME 



104 



ACTIVATE SCENE 
INDICATOR 




106 




ASSIGN 
DEFAULT CODE 



126 



LOG END OF 
LAST SCENE 



128 



LOG 
CODE 


i 




TURN I 
INDICAT 


SCENE 
OR OFF 



110 



112 




LOG END OF 
LAST SCENE 



STORE SCENE 
INDICIA 



RECORD ALL 
SCENE INDICIA 



END 



116 



118 




122 



FIG. 2 



2/2 



